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■^ I Abstract 

^H ' We present a local convergence analysis of inexact Newton-like methods for solving nonlin- 

r*^ . ear equations under majorant conditions. This analysis provides an estimate of the convergence 

"t^ ' radius and a clear relationship between the majorant function, which relaxes the Lipschitz con- 

tinuity of the derivative, and the nonlinear operator under consideration. It also allow us to 
obtain some important special cases. 
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(Ti ■ 1 Introduction 

(^ ■ Newton's method and its variations (see [4j), including the inexact Newton methods, are the most 

OO , efficient methods known for solving nonlinear equations 

o 

F{x) = 0, (1) 

p\ ' where F : $7 — > W^ is a continuously difFerentiable function and Q C M" is an open set. The inexact 

j^ ■ Newton method was introduced by Dembo, Eisenstat and Steihaug in [3j denoting any method 

which, given an initial point xq, generates the sequence {xk} as follows: 

Xk+i = Xk + Sk, k = 0,1,... , 

where Sk is the solution of the linear system 

F'{xk)Sk = -F{xk) + rk. 
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for a suitable residual r^ G M". Let x^ be a solution of ([T]) such that F'{x^) is invertible. As 
shown in [3J, if ||rfc|| < ^fc||-F(2;fc)|| for /c = 0, 1, . . . and {9k} is a sequence of forcing terms such that 
< 6**; < 1 then there exists e > such that the sequence {xk}, for any initial point xq £ B{x^, e) = 
{x G M" : ll^^, — x\\ < e}, is well defined and converges linearly to x* in the norm ||y||* = ||F'(x*)y||, 
where || || is any norm in R". It is worth noting that, in [3j, no Lipschitz condition is assumed on 
the derivative F' to prove that {xk} is well defined and linearly converging; however, no estimate 
of the convergence radius e is provided. As pointed out by [11] (see also [13]) the result of [3] is 
difficult to apply due to dependence of the norm || ||*, which is not computable. 

Formally, the inexact Newton- like methods for solving the non-linear equation ([1]), which we 
will consider, are described as follows: Given an initial point xq £ Q, define 

Xk+i= Xk + Sk, B{xk)Sk = -F{xk)+rk, A; = 0, 1,..., 

where B{xk) is a suitable invertible approximation of the derivative F'{xk) and the residual r^ 
satisfies 

WPkTkW <ek\\PkF{xk)l 

for suitable forcing sequence {Ok} and some invertible matrix sequence {Pk} of preconditioners for 
the above linear equation defining the steep Sk- This method was considered for the first time 
in [13], and was also considered in [10]. In particular, letting P^ = / be the identity matrix and 
Bk = F'{xk) for each k, we obtain the inexact Newton method considered in [3], ^JT\ and [19]. 

Inexact Newton-like methods may fail to converge and may even fail to be well defined. To 
ensure that the method is well defined and converges to a solution of a given non-linear equation, 
some conditions must be imposed. For instance, the classical convergence analysis (see [3]) requires 
the initial iterate to be "close enough" to a solution and the first derivative of the non-linear 
function to be invertible in this solution. Moreover, for estimating the convergence radius, the 
Lipschitz continuity or something like Lipschitz continuity, of the first derivative is also assumed 
(see [lO], [13] and [l9]). 

In the last twenty-five years, there have been papers dealing with the issue of convergence of 
the Newton methods, including the inexact Newton methods and the Gauss-Newton methods, by 
relaxing the assumption of Lipschitz continuity of the derivative (see [l] , [6] , [7] , [9] , [10] , [12] , [17] , 
|18j and [H]). In addition to improving the convergence theory (this allows us to estimate the 
convergence radius and to enlarge the range of application) some modifications of the Lipschitz 
condition also permit us to unify several results. Works dealing with this subject include UJ, |^, 
[7] and [18]. 

Our aim in this paper is to present a new local convergence analysis for inexact Newton-like 
methods under major ant condition. In our analysis, the classical Lipschitz condition is relaxed 
using a majorant function. It is worth pointing out that this condition is equivalent to Wang's 
condition introduced in [1^ and used by Chen and Li in [THI to study the inexact Newton-like 
methods. The convergence analysis presented is linear in an arbitrary norm. It provides a new 
estimate for the convergence radius and a clear relationship between the majorant function and the 



nonlinear operator under consideration. It also allows us to obtain some special cases that can be 
evaluated as an application. 

The organization of the paper is as follows. In Section 11.11 we list some notations and basic 
results used in our presentation. In Section [2] the main result is stated, and in Section 12.11 some 
properties involving the majorant function are established. In Section 12.21 we presented the re- 
lationships between the majorant function and the non-linear operator. In Section 12.31 the main 
result is proveda and some applications of this result are given in Section [3l Some final remarks 
are made in Section HI 

1.1 Notations and auxiliary results 

The following notations and results are used throughout our presentation. Let M" be with a norm 
||.||. The open and closed ball at a G M" and radius 6 > are denoted, respectively by 

B{a, 6) = {x£ M"; \\x - a\\ < 6}, B[a, 6] = {x € M"; ||x - a\\ ^ 6}. 

Let /^(M^jM") be the space of liner operators of M". Define the operator norm associated to the 
norm ||.|| as 

||r|| := supdlTxIl, ||x|| < 1}, Te £(M",M"). 

The condition number of an invertible operator T is denoted by cond(T) := ||T||||T~^||. 

Lemma 1 (Banach's Lemma). Let B G £(M",M") and I G £(IR",R") , the identity operator. If 
\\B - I\\ < 1, then B is invertible and \\B^^\\ < 1/ (1 - \\B - I\\) . 

Proof. See the proof of Lemma 1, pp. 189 of Smale [15] with A = I and c = \\B — I\\. D 

Proposition 2. IfO<t<l, then E*^o(^ + 2)(^ + 1)** = 2/(1 - tf. 

Proof. Take A: = 2 in Lemma 3, pp. 161 of Blum, Cucker, Shub and Smale [2]. D 

Also, the following auxiliary result of elementary convex analysis will be needed: 

Proposition 3. Let e > and r G [0, 1]. If ip : [0,e) ^M. is convex, then / : (0, e) — > M defined by 

is non-increasing. 

Proof. See Theorem 4.1.1 and Remark 4.1.2 on pp. 21 of Hiriart-Urruty and Lemarechal [8j. D 



2 Local convergence of inexact Newton-like method 

Our goal is to state and prove a local theorem for inexact Newton-like methods. Assuming that 
the non-linear equation F{x) = has a solution x*, we will, under mild conditions, prove that the 
inexact Newton-like method is well defined and that the generated sequence converges linearly to 
this solution. The statement of the theorem is as follows: 

Theorem 4. Let 0, C M" be an open set and F : i^ ^ M" a continuously differentiable function. 
Let x^: £ Q, R > and 

K := sup{i G [0, i?) : B{x^,t) C Q} . 

Suppose that -F(x*) = 0, F'{x^) is invertible and there exists a / : [0, i?) — > M continuously 
differentiable such that 

\\F'ix,)-^ [F'ix] - F'{x, + t{x - X,))] II < /' (||x - x,||) - /' (r||x - x,||) , (2) 

for T G [0, 1] and x G B{x^, k), where 

hi) /(O) = and f (0) = -1; 

h2) /' is convex and strictly increasing. 

Take < i? < 1, < W2 < "^i such that wit? + uj2 <1. Let u := sup{t G [0, R) : f'{t) < 0}, 

p:=sup{tG (0,1/) ■.uJi{l+^)[f{t)/{tf{t))-l]+uJi^ + uJ2 < 1}, a:=mm{K, p} . 

Then, the inexact Newton-like methods for solving F{x) = 0, with initial point xq G B{x^,a)\{x^:} 

Xk+i = Xk + Sk, B{xk)Sk = -F{xk) + rk, A; = 0, 1, . . . , (3) 

where B{xk) is an invertible approximation of F'{xk) satisfying 

\\B{xk)-^F'{xk)\\ < oJu \\B{xk)-^F'{xk) - L\\ < c^2, 

the residual r^ satisfies 

WPkTkW < ek\\PkF{xk)i (4) 

for some invertible matrix sequence {Pk} of preconditioners (for the linear system in ^) and a 
forcing sequence {9k} of non-negative numbers satisfying 

OkCondiPkF'ixk)) < T?, 

is well defined, contained in B{x^,,a), converges to x* and there holds 



|*^fc+i ^*|| _ 



xo-x^\\f'[\\xQ -x*||) 



I X/j^ **^* 1 1 ) "^ — U5 X , . . . . 



Note that letting the majorant function / : [0, k) ^ M be given by f{t) = Kt^ /2 — t, B{xk) = 
F'{xk), Pk = I , ioi = 1 and c<j2 = in TheoremlU we obtain the following local convergence result 
for the inexact Newton method: 

Theorem 5. Let il C M" be an open set and F : — > M" be continuously dijjerentiable in 0. Take 
x* G (7 and let k := sup{t > : B{x^,,t) C $1} . Assume that F'[x^) is invertible, F{x^) = 0, there 
exists a K > such that 

\\F'{x,)-^ [F'{x)-F'{y)]\\ <K\\x-y\\, V x,y G B(x*, k). 

Take < t? < 1. Let 

a := minJK, 2(1 - i?)/ {K{5 - i?))} . 

Then, the inexact Newton-like method for solving F{x) = 0, with the initial point xq G B{x^,a)\{x^} 
Xk+i = Xk + Sk, F'{xk)Sk = -F{xk)+rk, A; = 0,1,..., 

where the residual rk satisfies 

\\rk\\ <Ok\\F{xk)\\, 

for some forcing sequence {9k} of non-negatives numbers, satisfying 

9kCondF'{xk) < ^, 
is well defined, contained in B{x^:,a), converges to x* and there holds 



\xk+i-xA\ < 



K\\xo-x4 • 

^^ + ^^2(l-i^||xo-x,||)+^. 



\Xf^ X^ K V rC — Uj -L 5 . . . 



Remark 1. Note that letting i} = (in this case 9k = and rk = 0) in Theorem we obtain 
r = min{K, 2/{3K)}. As was shown in 116] (see also, ',20]) this is the best possible convergence 
radius for Newton's Method. Therefore, for vanishing residuals, Theorem merges into the theory 
of Newton's Method and, as a consequence, Theorem [7] does too. 

In order to prove TheoremlHwe need some results. From here on, we assume that all assumptions 
of Theorem H] hold. 

2.1 The majorant function 

Our first goal is to show that the constant k associated with and the constants v, p and a 
associated with the majorant function / are positive. Also, we will prove some results related to 
the function /. 

We begin by noting that k > 0, because is an open set and x* G i7. 



Proposition 6. The following statements hold: 
i) u>0; 

ii) /'(t)<0, yte[o,u); 
iii) The map [0, i') 3 1 1-^ l/\f'{t)\ is strictly increasing and 

w) t-f{t)/f'{t)<o, yte[o,u). 

Proof. As /' is continuous in (0, -R) and /'(O) = —1, there exists a 5 > such that f'{t) < for all 
t £ (0, 6). So, 1^ > 5 and item i is proved. 

For proving item ii, use h2 and the definition of i^. Now, for establishing the validity of item iii, 
combine h2 and item ii. 

Since /' is strictly increasing we have / is strictly convex. So, 

f{o)>f{t)-tf'{t), ytG{o,R). 

Because /(O) = and f'{t) < for all t G (0, v), the inequality in item iv follows from above 
equation. D 

Let rif be the Newton iteration associated with the majorant function, 

Uf : [0, ly) -^ (-00, 0] . 

t ^ t-f{t)/f'{t). ^^^ 

From Proposition [6] we have /' < in [0, v). Hence, the Newton iteration associated with the 
majorant function is well defined in [0, i'). 

Proposition 7. The map (0, u) 3 t h^ \nf{t)\/t'^ is strictly increasing. 

Proof. Using item iv of Proposition [6] and hi we obtain, after simple algebraic manipulation, that 

\nf{t)\ 1 f' fit) - f'JTt) ^ y.^rn , ff,, 

On the other hand, since /' is strictly increasing, we obtain that the map 

fit) - firt) 



(0, u) B t 



t 



is positive for all r G (0, 1). Also, from h2 we know that /' is convex. So, applying Proposition [3] 
with f' = ip and e = i^, we conclude that the last map is increasing. Hence the second term in the 
right hand side of ([6]) is positive and increasing. Therefore, since Proposition [6] implies that the 
first term in the right had side of dHD is positive and strictly increasing, we conclude the statement. 

D 
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Corollary 8. The map (0, v) B t ^^ \nf{t)\/t is strictly increasing. 

Proof. It is immediate, by noting that |nj(t)|/t = (|nj(t)|/t^)t is a product of two strictly increasing 
functions. D 

Proposition 9. The constant p is positive and there holds 

\nfit)\ 
uJi{l + 'dy-^^+uji'd + uj2<l, VtG(0, p). 

Proof. Using Proposition [6] and the definition (l5|), we have 

< f{t)/{tnt)) - 1 = [fit)/ fit) -t)/t= \nf{t)\/t, V t G (0, I.). (7) 

Now, because Proposition [7] implies that \nf^{t)\/t^ is bounded near zero, we obtain 

Jim \nf{t)\/t = \im{\nf{t)\/t^) t = 0. (8) 

Thus, since 1 — (wii? + W2)/wi(l + ■!?)> 0, using ^ and dH) we conclude that there exists a (5 > 
such that 

< {f{t)/{tf'{t)) - 1) < 1 - {uji'd + u;2)M(l + ^), V t G (0, 5), 

or, equivalently, 

0<c^i(l + ^)[/(t)/(t/'(t))-l]+c^i^ + L^2<l, VtG(0,(^). (9) 

Hence, combining the last equation and the definition of p, we have 6 < p, which is a proof of the 
first statement. 

For concluding the proof, we use the definition of p, equality ([9]), ([7]) and Corollary El D 

2.2 Relationship of the majorant function with the non-hnear operator 

In this section we will present the main relationships between the majorant function / and the 
non-linear operator F. 

Lemma 10. Let x £ ft. If \\x — x^:\\ < min{z^, k}, then F'{x) is invertible and 

\\F'{x)-'F'{x,)\\^l/\n\\x-xM- 

In particular, F' is invertible in B{x^,a). 

Proof. Let x £ fl such that ||x — 3;,,|| < min{i/, k}. So /'(||x — a;^,||) < which, together with ([2]), 
implies 

\\F'{x,)-^F'{x) - I\\ = \\F\x.r\F\x) - F'{xM < /'(Ik " ^*ll) " /'(O) < -/'(O) = 1. 



Thus, Lemma [T] and the last equation imply that F'{x^) ^F'{x) is invertible, as well as F'{x), and 

1 1 1 



"'(-r'F'Mll < r,ir(..)-^p.M-n ^ i 



(x.)-lF'(i) - /|| - 1 - (/'(III - x.W) - /'(O)) \f'(\\x - x.\\)\ ' 

where we assume that /'(O) = —1 and /' < in [0, z/) in the last equality. As o" < z^ the last part 
is proved. D 

The Newton iteration at a point happens to be a zero of the linearization of F at such point, 
which is also the first-order Taylor expansion of F. So, we study the linearization error at point in 

n 

EF{x,y):=F{y)-[F{x)+F'{x){y-x)], y, x E 0. (10) 

We will bound this error by the error of the linearization of the major ant function / 

ef{t,u):=f{u)-[f{t) + f'{t)iu-t)], t,ue[0,R). (11) 

Lemma 11. /f ||x=k — x\\ < n, then there holds ||-F'(x*)~-'-i?ir(x,x*)|| < ej(||x — x*||,0). 

Proof. Since B{x^, k) is convex, we obtain that x* + t{x — x*) G -B(x*, k), for < r < 1. Thus, as 
F is continuously differentiable in 0, the definition of Ep and some simple manipulations yield 

\\F'{x^)-^Ef{x,x^)\\< I \\F'{x^)-\F'{x)-F\x^ + t{x~x^))\\\ ||x, -x|| dr. 

Jo 

From the last equation and the assumption ([2|), we obtain 

\\F'{x,)-'Ef{x,x,)\\< [ [/'(||x-x*||)-/'(t||x-x*||)] ||x-x*||dT. 

Jo 

Evaluating the above integral and using the definition of ej, the statement follows. D 

Define the Newton step to the functions F and / by the following equalities: 

5^(x) := -F'ix)-^Fix), Sf{t) := -/(t)//'(t). (12) 

Lemma 12. // ||x — x=k|| < min{i/, k}, then ||5i?(x)|| < s/(||x — x*||). 

Proof. Using (|12p . -F(x*) = and some algebraic manipulation, it follows from (jlOp that 

||5f(x)|| = II - F'(x)-i (F(x,) - [F{x) + F'(x)(x, - x)]) + (x, - x)|| 

< ||F'(x)~iF'(x,)|| ||F'(x,)-i (F(x,) - [F(x) + F'(x)(x, - x)]) || + ||x, - x|| 
= ||F'(x)-1f'(x,)||||F'(x,)-^^f(x,x,)|| + ||x-x*||. 



Combining the last equation with Lemma [10] and Lemma [TT] we have 

lie I Ml ^ e/(lk-a:^*lliO) , II II 

Since /' < in [0, i^) and ||x — x=k|| < i^, we obtain from last inequality (jlip and hi, that 

no . Mi^ /(0)-/(lk-^*ll)+/(lk-^*ll)lk-^*ll , II II _ /(lk-^=.||) 

"^^^"^" - -/'(llx-x.ll) + Ik - ^4 - ;.(||^_^^||)- 

So, the last inequality together with the second equality in (fT2|) implies the desired inequality. D 



Lemma 13. Let il C M" he an open set and F : 0, ^ M" a continuously differentiahle function. 
Let X* € i7, i? > and k := sup{t G [0, R) : B{x^,,t) C fi} . Suppose that F{x^) = 0, F'{x^) is 
invertible and there exists a f : [0, R) ^ M continuously differentiahle satisfying ^, hi and h2. 
Let-d, ijj2, uJi, u, p and a as in Theorem^ Assume that x £ i?(x*,o")\{x*}, i.e., < ||j; — x*|| < a. 
Define 

x+ = x + S, B{x)S = -F{x)+r, (13) 

where B{x) is a invertible approximation of F'{x) satisfying 

||B(x)"^F'(x)|| < LOi, \\B{x)-'^F'{x) - L\\ < L02, (14) 

and that the residual r satisfies 

||Pr|| < 6l||PF(x)||, (15) 

for some 9 and P non-negative number and invertible matrix, respectively, satisfying 

econd{PF'{x)) < ^, (16) 

then x+ is well defined and there holds 



r < 



T — T" 



\>h Xri: 



In particular, 

1 1 X_|_ -^^ 1 1 "^ 1 1 -J^ -^^ 1 1 • 

Proof. First note that, as ||a; — 3;*|| < t < o", it follows from Lemma [TO] that F'[x) is invertible. 
Now, let B{x) a invertible approximation of it satisfying (I14p . Thus, x+ is well defined. Now, as 
F{x^) = 0, some simple algebraic manipulation and ()13p yield 

x+ - X* = B{x)-^ (F(x*) - [F(x) + F'(x)(x* - x)]) + {B{x)-^F'{x) - /)(x* - x) + B{x)-^r. 



So, the above equation and (fTO|) give 

x+-x^ = B{x)-^Ef{x, x^) + {B{x)-^F'{x) - /)(x, - x) + B'^{x)r. 

Again, some algebraic manipulation in the above equation, together with the properties of the 
norm, imply 

||x+ - x,|| < \\B{x)-'^F'{x)\\\\F'{x)-^F'{x^)\\\\F'{x^)'^Ef{x,x^)\\ 

+ \\B{x)-^F'{x) - I\\\\x - x4 + \\B{xy^F'{x)\\\\F'{x)-^p-^\\\\Pr\\. 



Taking into account the assumptions (J14j) e (llSp we obtain from the last equation that 

||x+ — x*|| < uji\\F'{x)~^F'{x^)\\\\F'{x^)^^Ef{x,x^)\\ + UJ2\\x — X^:\\ 

+ uJie\\F'{xy^p-^\\\\PF{x)\\. 

On the other hand, ([l6j) implies 6'||(PF'(a;))"^||||PF'(x)|| < i?. So, it is easy to see from ([H]) that 
ujie\\F' {x)-^p-^\\\\P F{x)\\ < uJie\\{P' F{x))'^\\\\PF' {x)\\\\Sf{x)\\ < uji^Sf{x)\\. 

Hence, it follows from the two latter equations that 

||x+-x*|| <uji\\F'{x)-^F'{x^)\\\\F'{x^)-'^Ef{x,x^)\\ +W2||x-x*|| +uji^Sf{x)\\. 

Combining the last equation with Lemma [TOl Lemma [11] and Lemma [12] we conclude that 
II II . e/(||x-x^||,0) 

\\X+ -X^W < LOi ' _ -p— +U)2\\X-X^\\ +U)l'dSf[\\x-X^\\). 

Now, using (fTT|) . (I5|), hi and (fT2|) we have, by direct calculus, 

\fi{\\x-*x \\)\ " l'^/(ll^~^*ll)l' Sf{\\x-x*\\) = \nf{\\x-x4)\ + ||x-x*||. 

Therefore, it follows from above inequality and the two latter equalities that 

||x+ — x*|| < LiJi|nj(||x — x*||)| +a;2||2; — x*|| + ooid (|nj(||x — x*||)| + ||x — x*||) , 

which is equivalent to the first inequality of the lemma. 

Because x € B{x^,,a)\{x^,}, i.e., < ||x — x*|| < a we obtain the last inequality of the lemma 
by combining the first one and Proposition [9] with t = ||x — x=k||. D 
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2.3 Proof of Teorem d 

We are now in position to prove Theorem [H 

Proof. Since xq E -B(x*, a)\{x^}, i.e., < ||xo — x=k|| < o", a straighforward induction argument and 
the last inequahty in Lemma [T3] implies that the sequence {xk} generated by inexact Newton- like 
methods is well defined and contained in B{x^,, a). 

Our task is now to show that {xk} converges to x^. Because, {xk} is well defined and contained 
in B{x^,a), applying Lemma [13] with x-^. = Xk+i, x = Xk, r = r^, B{x) = B{xk), P = Pk and 
9 = 9k we obtain 



|2^A;+l ~ 27*11 < 



L^l(l+l9) 



\nf{\\Xk -x*| 






+ cJi^ + a;2 



|X^ x^ II , 



A: = 0,1,.... 



(17) 



A; = 0,1,... 



In particular, the last inequality implies that ||xfe_(_i — 2;^,|| < ||xo 
together with ()17p and Corollary [U gives 



X* II, for A; = 0, 1, ... , which. 



|*^A:+l ^*|| _ 



a;i(l + ^) 



|n/(||xo - x^\ 

1 1 "^0 "^^ 1 1 



+ UJl'& + L02 



I '^k •^^ II 5 



0,1,.... 



(18) 



As Proposition [9] with t = \\xo — x*|| gives ciJi(l + '!?)|n/(||xo — 2;*||)|/||xo — x*|| + ciJi'!? + uj2 < I, we 
conclude from above equation that {||xfc — x*||} converges to zero. So, {x^} converges to x^. 

It remains to prove the last inequality of the theorem. For this, use (llSp and the definition 
in (p. D 

Remark 2. If a continuously differentiable function / : [0, k) ^M is a majorant function satisfying 
the conditions hi and h2, then the function h : (— k, k) ^ M zs defined by 



hit) 



-fi-t), tG{-K,0], 
fit), t G [0, k). 



(19) 



satisfies all hypotheses of Theorem^ Indeed, it is straightforward to show that /i(0) = 0, /i'(0) = — 1, 
h'it) = f'i\t\) and that 



\h'iO)-^[h'it)-h'iTt)]\<f'i\t\)-fiT\t\), 



T e [0,1], t E (-K, k). 



So, F = h, n = 1, n = (— K, K.) and x* = satisfy all hypotheses of Theorem^ Therefore, we can 
apply Theorem^ to solve hit) = 0. 

Note that if f is not Lipschitz, then h' is also not Lipschitz. Therefore, we conclude that 
Theorem [^ enlarges the range of application of theorems on inexact Newton-like Methods having 
the Lipschitz condition as a hypothesis on the first derivative of the non-linear operator under 
consideration. 
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Now, we will give some examples of majorant functions satisfying the conditions hi and h2 
with a first derivative that is not Lipschitz. 

Example 1. The following functions satisfy the conditions hi and h2; 

i) f : [0, +00) -^ R such that f{t) = e* - 2i - 1; 

a) / : [0, k) ^ M such that f{t) = -(1/k) ln(l - nt) - 2t; 

Hi) / : [0, k) ^ M such that f{t) = [(1 - Kt)/K\ ln(l - nt). 

Note that the first derivatives of each of the functions above are not Lipschitz. 

Remark 3. The assumption ([2]) was crucial for our analysis. It is worth pointing out that, under 
appropriate regularity conditions on the nonlinear operator F , the assumption ([2]) always holds 
in a suitable neighborhood of x* . For instance, if F is twice continuously differentiable, then the 
majorant function / : [0, At) — > R defined by f{t) = Kt'^/2 — t, where K = sup{||F'(x*)"-'^F"(x)|| : 
X G B[x.f^K)} satisfies the assumption ([2]). Estimating the constant K is a very difficult problem. 
Therefore, the goal is to identify classes of nonlinear operators for which it is possible to obtain a 
majorant function. We will give some examples of such classes in the next section. 

3 Special cases 

In this section we present three special cases of Theorem HI Namely, convergence results under 
an affine invariant Lipschitz condition, Smale's condition for analytical functions and Nesterov- 
Nemirovskii's condition for self-concordant functions. 

3.1 Convergence result for afRne invariant Lipschitz condition 

In this section we show a correspondent theorem to Theorem [4] under an affine invariant Lipschitz 
condition (see [5] , [9] and [13] ) instead of the general assumption ^ . 

Theorem 14. Let Q C M" be an open set and F : fi ^ R" be continuously differentiable in Q. 
Take x^, £ il. and let 

K := sup {t > : B{x^,t) C Vt] . 

Assume that F'(x*) is invertible, F(x*) = 0, and there exists a K > such that 

\\F'{x,)-^[F'{x)-F'{y)]\\<K\\x-y\\, y x,y e B{x^,k). (20) 

Take < f? < 1, < W2 < 1^1 such that loii} + uj2 < 1- Lst 

a := min {k, 2(1 - t?wi - W2)/ {K{2 + wi - -duji - 2^2))} • 
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Then, the inexact Newton-like method for solving F{x) = 0, with an initial point xq G B{x^,a)\{x^} 

Xk+i = Xk + Sk, B{xk)Sk = -F{xk) + rk, k = 0,l,..., (21) 

where B[xk) is an invertible approximation of F'{xk) satisfying 

\\B{xk)-^F'{xk)\\ < uji, \\B{xk)-^F'{xk) - I\\ < UJ2, 

and the residual r^ satisfies 

WPkrkW < Ok\\PkF{xk)\\, (22) 

for some invertible matrix sequence {Pk} of preconditioners and forcing sequence {9k} of non- 
negatives numbers, satisfying 

9kCond{PkF'{xk)) < ^, 

is well defined, contained in B{x:^,a), converges to x* and there holds 



\'^k~\'l ^*|| _ 



^l(l + t})7^^ r^ ^ + Wit? + U2 

2(1 - A||a;o - x*||) 



\Xf^ X^ 5 V fC — U, -L , . . . . 



Proof. It is immediately possible to prove that F, x* and / : [0, k) ^ M defined by f{t) = Kt^ /2 — t, 
satisfy the inequahty ([2]) and the conditions hi and h2 in Theorem [H In this case, it is easy to see 
that the constants p and z^, as defined in Theorem [H satisfy 

P = 2{1- -duJi - uj2)l {K{2 + uJi- i)uJi - 2uJ2)) <v = l/K, 

as a consequence 

a := mm{K, 2(1 - ^uji - wa)/ {K{2 + uji - ^uJi - 2^2))}- 

Therefore, as F, a, f and x* satisfy all of the hypotheses of Theoremdl taking xq £ B{x^, '^)\{^*} 
the statements of the theorem follow from Theorem HI D 

Although the condition (j20p is affine invariant (it is insensitive with respect to transformation of 
the map F of the form F 1— > AF) , iteration (j2ip and the condition for the residual (j22|) is not affine 
invariant. So, Theorem [TJ] is not affine invariant. Now, taking B{xk) = F'{xk) in iteration ()2ip 
and Pk = F'{xk)~^ in the condition for the residual (j22p . the Theorem 1141 becomes affine invariant. 
It is easy to see that, for the theorem that uses the Lipschitz condition 

||F'(x)-F'(y)|| <L||2;-y||, y x,y e B{x,,k), 

instead of the affine invariant Lipschitz condition (j20p , the convergence radius is given by 

a := min {k, 2(1 - i?wi - UJ2)/ (L||F'(x,)||(2 + loi - ^uJi - 2u;2)) } . 

We point out that the convergence radius of affine invariant theorems are insensitive to invertible 
transformation of the map F, but that theorems with the Lipschitz condition (see next example) 
are sensitive. For more details about affine invariant theorem see [5]. 
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Example 2. Assume that B{xk) = F'{xk), Pk = F'{xk) ^ , u)i = 1 and u)2 = in Theorem\14\ Let 
F : R2 ^ M^ 5g gj^y^^ f^y F{xi,X2) = {xj/2 - xi, x\l2 - X2). Note that F(0,0) = (0,0). Using the 
Euclidean vector norm and the associated operators norm, it is easy to see that ||i^"(2;i,X2)|| = 1 
and the Lipschitz constant for F' is 1. In this case, the Lipschitz condition for F' and the affine 
invariant Lipschitz condition ()20p are equal. Therefore, by applying Theorem [I^ we conclude that 
the convergence radius for solving F{xi, X2) = is 2(1 — t?)/ {3 — 1}) . Let the invertihle matrix 



and the map G : Q 



A = 

V^ given by G{x) 
l|G"(0,0)-i| 



1 ■ 

1/e 



0< e< 1, 



AF{x). Hence G'{xi,X2) = AF' {xi,X2)- Moreover, 
= 1, ||G"(xi,X2)|| = l/e, 



and the Lipschitz constant for G' is 1/e. Applying Theorem^T^ with B{xk) = G'{xj^) and P^ = 
G'{xk)~^, we conclude that, due to its insensitivity to invertible transformation, the convergence 
radius for solving G{xi,X2) = is also 2(1 — tD)/ (3 — ■!?). However, if in Theorem I4 the Lips- 
chitz condition for F' is assumed instead of the affine invariant Lipschitz condition ()20p . then the 
convergence radius is (2e)(l — ■!?)/ (3 — "!?). 

3.2 Convergence result under Smale's condition 

In this section we show a correspondent theorem to Theorem H] under Smale's condition. For more 
details about Smale's condition see |15j. 

Theorem 15. Let C M" be an open set and F : Q ^ M" an analytic function. Take x^ £ il. such 
that -F'(x*) is invertible and F(a;*) = and let 



K := sup{t > : B{x^,t) C Q.} and 7 := sup 

n>l 



F'(x*)-1fW(x,) 



n\ 



l/(n-l) 



< +00. (23) 



Take < ?? < 1, < W2 < "^i such that loi'& + L02 < 1. Let a = 1^1(1 + iD), 6 = (1 — uji^ — UJ2) and 



a := mm < k, 



+ 46 - (v^(a + 46)2 - 862) 
467 



Then, the inexact Newton-like method for solving F{x) = 0, with initial point xq £ B{x^, o')\{Xif} 

Xk+i = Xk + Sk, B{xk)Sk = -F{xk) + rk, A; = 0,1,..., 

where B{xk) is an invertible approximation of F'{xj.) satisfying 

\\B{xk)-^F'{xk)\\ < uji, \\B{xk)-^F'{xk) - I\\ < UJ2, 
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and the residual r^ satisfies 

WPkrkW <ek\\PkF{xk)\l 

for some forcing sequence {0^} of non-negative numbers and an invertihle matrix sequence {Pk} of 
preconditioners, satisfying 

9kCond{PkF'{xk)) < ^, 

is well defined, contained in B{x^,,a), and converges to x* 



l-^/c+l -^^ II _ 



.-'" +-" 2(1 -;!::::!)^-i ""■"""- 



\xk — a;*||, V A; = 0, 1, 



We need the following result to prove the above theorem. 



Lemma 16. Let Vi C M" he an open set and F : 0, ^ M" an analytic function. Suppose that 
X* G $7, F'{x^) is invertihle and that B{x^,,l/j) C il, where 7 is defined in (I23p . Then, for all 
x £ B{x^, 1/7) there holds 

||F'(x,)-^F"(x))K (27)/(l - 7lk - x,||)3. 
Proof. Let x G $7. Since F is an analytic function, we have 



F\x,)-^F"{x) = y 1f'(x*)-1f("+2)(^^)(^ _ 3.^)n^ 

n=0 

Combining (|23p and the above equation we obtain, after some simple calculus, that 

00 
\\F'{x,r^F"{x)\\ ^ 7^(n + 2)(n + l)(7||x-x,||r. 



n=0 



On the other hand, as i?(x*, I/7) C we have 7||x — x*|| < 1. So, from Proposition [2] we conclude 

2 



J](n + 2)(n + l)(7||= 



(l-7||x-x=,||)3 ^^^ 
Combining the two above equations, we obtain the desired result. D 

The next result gives a condition that is easier to check than condition ([2]), whenever the 
functions under consideration are twice continuously differentiable. 

Lemma 17. Let Q. C M" he an open set and F : Vt ^ Y he twice continuously on il. Let x* G O 

with F'(x*) he invertihle. Lf there exists a / : [0,i?) — > R twice continuously differentiahle such that 

||F'(x,)-iF"(x)Kr(||x-x.||), (24) 

for all X £ 0, such that \\x — x*|| < R. Then F and f satisfy ([2]). 
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Proof. Taking r E [0, 1] and a; G O, such that x^, + t{x — x^) £ Q and ||a; — x^,|| < R, we obtain that 

\\F'{x^)-^ [F'{x) - F'{x^ + t{x - X,))] II < / \\F'{x^)-'^F"{x^ + t{x - x,))|| ||x - x,||dt. 
Now, as llx — x*|| < R and / satisfies (|24p . we obtain from the last inequality that 

\\F'{x^)-^[F'{x)-F'{x^ + t{x-x^))]\\< f /"(t||x-x*||)||x-x*||dt. 

Evaluating the latter integral, the statement follows. D 

[Proof of Theorem 115] . Assume that all hypotheses of Theorem [15] hold. Consider the real 
function / : [0, 1/7) -^ M defined by 

fit) = — 2t. 

It is straightforward to show that / is analytic and that 

/(0) = 0, /'(t) = l/(l-7t)'-2, /'(0) = -l, r(t) = (27)/(l-7t)', r(0)=n!7"-\ 

for n > 2. From the last four equalities it is easy to see that / satisfies hi and h2. Now, since 
f"{t) = (27)/(l — 7t)^ combining Lemma [T71 Lemma [T6l we conclude that F and / satisfy ([2|) with 
R = 1/7. Define 

V := sup{t G [0,1/7) : f'{t) < 0}, p := sup{t e (0, zv) : uJi{l + '&)[f{t)/{tf'{t))-l]+uJi'& + U2 < 1}. 

In this case, it is easy to see that the constants v and p satisfy 

a + 46-(V(a + 46)2-862) \/2 - 1 1 

46 V27 7 

where a = u)i{l + i9), b = (1 — uii} — a;2)- Finally, let a := minJK, p}. Therefore, as F, a, f and 
X* satisfy all hypothesis of Theorem [U taking xq € i3(x*, (t)\{x*}, the statements of the theorem 
follow from Theorem [Jl D 

3.3 Convergence result under The Nesterov-Nemirovskii condition 

In this section we show a correspondent theorem to Theorem [1] under the Nesterov-Nemirovskii 
condition(see p4]). 

Let Q C M" be a convex set. A function g( : il ^ M is called a-self-concordant with the parameter 
a > 0, if 5 G C^{Q,) , i.e., three times continuously differentiable in fi, is a convex function on J7 
and satisfies the following inequality 

\g"'{x)[h,h,h]\ ^2a-^/^{g"{x)[h,h]f/^, V x G f], V /i G M". (25) 
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Take x^, G il such that g"{x^) is invertible. Define X := (M", (., .)j,.^) as the Euchdean space 
with the inner product and the associated norm defined, respectively, by 



{u,v)x,:=a ^{g"{x^)u,v), \\u\\x, := ^/(u~uj~J~, 

where (., .) is the Euchdean inner product. So, the open and closed ball of radius r > centered 
at x^ ( Dikin's ellipsoid of radius r > centered at x* ) in X are defined, respectively, as 

Wr(x*) := {x G M" : ||x - x*||a;» < r} , l^ri^;*] := {x G M" : ||x - x*||a;, < r} . 

Theorem 18. Let Q (1 X be a convex set and g : fi — > R an a- self- concordant function. Take 
X* G ri with g"(x^) invertible and let k := supju > : VF„(x=k) C i7}. Suppose that g'{x^) = 0. 
Take < ?? < 1, < 0^2 < wi such that ujii!} + uj2 < 1. Let a = loi{1 + •&) , 6 = (1 — uiii9 — 0^2) and 



a + ib- (\/(a + 46)2 - 862) 
a:=mm|., 1. 

Then, the inexact Newton-like method for solving g' (x) = 0, with an initial point xq G B{Xif,a)\{x^,} 

Xfc+i = Xfc + 5'fc, 5(xfc)5fc = -F(xfc) + rfc. A; = 0,1,..., 

where B{xk) is an invertible approximation of g"{xk) satisfying 

\\B{xk)~^g"{xk)\\ < uji, WBixky^g'ixk) - I\\ < ^2, 
and the residual r^ satisfies 

WPkTkW <ek\\PkF{xk)l 

for some forcing sequence {Ot} of non-negative numbers and an invertible matrix sequence {Pk} of 
preconditioners, satisfying 

9kCond{PkF'{xk)) < ^, 

is well defined, contained in B[x^,a), converges to x* and 



\Xk-\-\ X^W f^ 



"■""'" 2(i-!::::!)^-i +-'^+'-'. 



X^ X:^ K V rC — U5 -L J . . . . 



We need some auxiliary results about self-concordant functions to prove the above theorem. We 
begin with two well known propositions in the theory of self-concordant functions, from Nesterov 
and Nemirovskii [14J. 

Proposition 19. Let Vt d X be an open convex set and let g : Q ^ M. be an a- self- concordant 
function. Then, 

\g"'{x)[hiMM]\ ^2a-^/^Y]^,^^{g"{x)[hiMf'\ V x G S7, Vh^MM^X. 
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Proof. See Proposition 9.1.1, Appendix 1, pp.361 of [H]. D 

Proposition 20. Let il. C X be an open convex set and let g : Q ^ M be an a- self- concordant 
function. Assume that Wi{x^) C VL. Then there holds 

g"{x)[h,h]< \ g"{^:j,^)[h^h], VxGVFi(x,), V/iGX 

yi \\x x^ 11^* / 

Proof. See Theorem 2.1.1 pp.13 of [T3]- D 

The next result is a combination of the two last propositions, which has appeared in [Ij 
Lemma 5.1. We include the proof here. 

Lemma 21. Let 0, C X be an open convex set and let g : Q ^M be an a- self- concordant function. 
Assume that Wi{x*) C il. Then 

\\9"{x.)-^9"'{x)\\^, < I V X G W,{x,). 

yl \\X ^*||a:*/ 

Proof. Letting x £ Wi{x^) and hi, /i2, /is G K" we have from ()25p and Proposition [T9l 

\{g"{x^y^g'"{x)hih2,h3)x^\ = a~^\{g"{x^) [g" {x^y^ g" {x)) /ii/i2,/i3)| 

= a-^\g"'{x)[hiMM]\ 

2a-'"^W,^i{g"{x)[h,Mf/\ 



< 



Since \\g"ix^) ^g"'{x)\\x,^ := sup {|(5"(x*) ^g"'ix)hih2,h3)xj : \\hi\\x^ s^ 1, i = 1,2,3} , we have 
from the last inequality 

\\g"{x,)-^g"'ix)\\x, <2a-3/2sup{nii(/(x)[/ii,/i,])i/2 : \\h,\\,^ < 1, i = 1,2,3}. (26) 
Therefore, it follows from (|26p and Proposition 1201 that 

O/j— 3/2 , -^ 

lb"(x*)-^/'(x)|U, < sup \uU9"ix*)[hi,hi])'/^ : \\h,\U < 1, i = l,2,3l 

(1 - \\X - X^:\\xJ'^ ^ J 

— 3Sup{^3^la-l/V(a:*)[/^^,/l^])'/':||/l^lU., < 1, ^ = 1,2,3) 



< 



[i \\X X^:\\Xf 

2 

yi \\X X^: \\xf ) 

2 



sup{nf^i||/ij||a;^ : \\hi\\x, <l, i = 1,2,3} 



/-I II II \3 ' 

which is a proof of the Lemma. D 
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[Proof of Theorem 118] . Assume that ah hypotheses of Theorem [15] hold. Consider the real 
function / : [0, 1) — > M defined by 

It is straightforward to show that / is analytic and that 

/(0) = 0, /'(t) = l/(l-t)2-2, /'(0) = -l, fit) = 2/(1 -tf, r(0)=n!, 

for n > 2. From the last four equalities, it is easy to conclude that / satisfies hi and h2. Now, 
combining Lemma [iTl Lemma [21] and the latter equality we obtain that g' and / satisfy ([2]) with 
R = l. Define 

1^ := sup{t G [0, 1) : /'(t) < 0}, p := sup{t G (0, i^) : a;i(l + i9)[/(t)/(t/'(t)) - 1] + a;ii9 + i02 < !}■ 

In this case, it is easy to see that the constants v and p satisfy 



a + 46-(v^(a + 46)2-862) ^/2 - 1 

P= 4^ , "=^7r' ^<"<'' 

where a = a;i(l + -(?), b = (1 — ujii} — 102)- Finally, let a := minJK, p}. Therefore, as -F = g', a, f and 
X* since the above satisfy all hypotheses of Theorem [H taking xq E B{x^,a)\{x^}, the statements 
of the theorem follow from Theorem [J] D 

4 Final remarks 

As pointed out by Morini in [13j if preconditioning P^, satisfying 

\\Pkrk\\<0k\\PkF{xk)\\, (27) 

for some forcing sequence {0fc}i is applied in finding the inexact Newton steep, then the inverse 
proportionality between each forcing term 9k and cond{PkF' (xk)) stated in the following assump- 
tion: 

<9kCond{PkF'{xk))<'&, A; = 0,1,..., (28) 

is sufficient to guarantee convergence, and may be overly restrictive to bound the sequence {9k}, 
always such that the matrices PkF'{xk), for A: = 0, 1, ... , are badly conditioned. Moreover, 9k does 
not depend on cond(Pfc) but only on the coiid{PkF' (xk)) and a suitable choice of scaling matrix Pk 
leads to a relaxation of the forcing terms. 

Using the assumptions ()27p and (I28p . we presented a new local convergence analysis for inexact 
Newton-like methods under majorant condition. In our analysis, the affine invariant Lipschitz 
condition (see [5], [9] and [E]) is relaxed by using the majorant condition (see equation ([2|) in 
Theorem [3]). Although the condition ([2|) is equivalent to the Chen and Li condition (see equation 
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(1.4) in [lOj), our analysis elucidates the relationship of the majorant function with the non-linear 
operator under consideration (see Lemma [TOl In addition, Lemma [TT| Lemma [12] and Lemma \T3\i 
allow us to obtain the special cases Theorem [TH Theorem [15] and Theorem [18] of Theorem H] as an 
application. 

Finally, we point out that the Kantorovich analysis produced a semilocal convergence result, 
in that it ensures convergence of Newton's Method under very mild assumptions and proves the 
existence of a solution. On the other hand, local analysis gives us the optimal convergence radius. 
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